Predictive and Multi-rate Sensor-Based Planning under Uncertainty

Email Print Request Permissions In this paper, a general formulation of a predictive and multirate (MR) reactive planning method for intelligent vehicles (IVs) is introduced. The method handles path planning and trajectory planning for IVs in dynamic environments with uncertainty, in which the kinodynamic vehicle constraints are also taken into account. It is based on the potential field projection method (PFP), which combines the classical potential field (PF) method with the MR Kalman filter estimation. PFP takes into account the future object trajectories and their associated uncertainties, which makes it different from other look-ahead approaches. Here, a new PF is included in the Lagrange-Euler formulation in a natural way, accounting for the vehicle dynamics. The resulting accelerations are translated into control inputs that are considered in the estimation process. This leads to the generation of a local trajectory in real time (RT) that fully meets the constraints imposed by the kinematic and dynamic models of the IV. The properties of the method are demonstrated by simulation with MATLAB and C++ applications. Very good performance and execution times are achieved, even in challenging situations. In a scenario with 100 obstacles, a local trajectory is obtained in less than 1 s, which is suitable for RT applications

Kinematic Control System for Car-Like Vehicles

Abstract. In this paper, we have highlighted the importance of the WMR model for designing control strategies. In this sense, the differential model has been used as reference model in order to design the control algorithm. After the control has been design, new actions will be generated for each additional wheel of the real vehicle (non-differential model). This new approach simplifies the overall control systems design procedure. The examples included in the paper, illustrate the more outstanding issues of the designed control. Moreover, we have particularized this control for the line tracking based on a vision system. A velocity control in the longitudinal coordinate has been implemented instead of a position control because we have no longitudinal information. Also, we have simulated and validated this control, studying the effect of the sampling time on the WMR behavior

Probabilistic Self-Localization and Mapping: An Asynchronous Multirate Approach

"© 2008 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works."[EN] In this paper, we present a set of robust and efficient algorithms with O(N) cost for the solution of the Simultaneous Localization And Mapping (SLAM) problem of a mobile robot. First, we introduce a novel object detection method, which is mainly based on multiple line fitting method for landmark detection with regular constrained angles. Second, a line-based pose estimation method is proposed, based on LeastSquares (LS). This method performs the matching of lines, providing the global pose estimation under assumption of known Data-Association. Finally, we extend the FastSLAM (FActored Solution To SLAM) algorithm for mobile robot self-localisation and mapping by considering the asynchronous sampling of sensors and actuators. In this sense, multi-rate asynchronous holds are used to interface signals with different sampling rates. Moreover, an asynchronous fusion method to predict and update mobile robot pose and map is also presented. In addition to this, FastSLAM 1.0 has been also improved by considering the estimated pose with the LS-approach to re-allocate each particle of the posterior distribution of the robot pose. This approach has a lower computational cost than the original Extended Kalman Filtering (EKF) approach in FastSLAM 2.0. All these methods have been combined in order to perform an efficient and robust self-localization and map building process. Additionally, these methods have been validated with experimental real data, in mobile robot moving on an unknown environment for solving the SLAM problem.This work has been supported by the Spanish Government (MCyT) research project BIA2005-09377-C03-02 and by the Italian Government (MIUR) research project PRIN2005097207.Armesto, L.; Ippoliti, G.; Longhi, S.; Tornero Montserrat, J. (2008). Probabilistic Self-Localization and Mapping: An Asynchronous Multirate Approach. IEEE Robotics & Automation Magazine. 15(2):77-88. https://doi.org/10.1109/M-RA.2007.907355S778815

Haptic Feedback to Assist Bus Drivers for Pedestrian Safety at Low Speed

Buses and coaches are massive Passenger Transportation Systems (PTS), because they represent more than half of land PTS in the European Union. Despite of that, bus accident figures are lower than other means of transport, but its size and weight increase the severity of accidents in which buses are involved, even at low speed. In urban scenarios, turnings and manoeuvres around bus stops are the main causes of accidents, mostly due to low visibility, blind spots or driver s distractions. Therefore, there is an increasing interest in developing driving assistance systems to avoid these situations, among others. However, even though there are some solutions on the market, they are not meant to work in urban areas at low speed and with the sole purpose of preventing collisions with pedestrians. In this sense, the paper proposes an active safety system for buses in manoeuvres at low speed. The safety system consists of haptic feedback devices together with collision avoidance and risk evaluation systems based on detected people nearby the bus. The performance of the active safety system has been validated in a simulated urban scenario. Our results show that driver s reaction time is reduced and time to collision increased due to the proposed low-speed active safety system. In particular, it is shown that there is a reduction in the number of high risk cases and collisions, which implies a considerable improvement in safety terms. In addition to this, a brief discussion about current regulations for innovative safety systems on a real vehicles is carried out.This paper has been funded by Ministerio de Ciencia e Innovacion (Spain) through the projects "Sistemas Avanzados de Seguridad Integral en Autobuses (SAFEBUS)" (IPT-2011-1165-370000) and "Sistemas de Conduccion Segura de Vehiculos de Transporte de Pasajeros y Materiales con Asistencia Haptica/Audiovisual e Interfaces Biomedicas (SAFETRANS)" (DPI2013-42302-R). This work was also supported by Programa VALi+d (Generalitat Valenciana). The authors wish to thank Jose Luis Sanchez Carrascosa for his dedication and commitment to the project and thank to Ana Isabel Sanchez Galdon for her valuable help regarding ANOVA analysis.Girbés, V.; Armesto Ángel, L.; Dols Ruiz, JF.; Tornero Montserrat, J. (2016). Haptic Feedback to Assist Bus Drivers for Pedestrian Safety at Low Speed. IEEE Transactions on Haptics. 9(3):345-357. https://doi.org/10.1109/TOH.2016.2531686S3453579

On the detection of defects on specular car body surfaces

[EN] The automatic detection of small defects (of up to 0.2 mm in diameter) on car body surfaces following the painting process is currently one of the greatest issues facing quality control in the automotive industry. Although several systems have been developed during the last decade to provide a solution to this problem, these, to the best of our knowledge, have been focused solely on flat surfaces and have been unable to inspect other parts of the surfaces, namely style lines, edges and corners as well as deep concavities. This paper introduces a novel approach using deflectometry- and vision-based technologies in order to overcome this problem and ensure that the whole area is inspected. Moreover, since our approach, together with the system used, computes defects in less than 15 s, it satisfies cycle time production requirements (usually of around 30 s per car). Hence, a two-step algorithm is presented here: in the first step, a new pre-processing step (image fusion algorithm) is introduced to enhance the contrast between pixels with a low level of intensity (indicating the presence of defects) and those with a high level of intensity (indicating the absence of defects); for the second step, we present a novel post-processing step with an image background extraction approach based on a local directional blurring method and a modified image contrast enhancement, which enables detection of defects in the entire illuminated area. In addition, the post-processing step is processed several times using a multi-level structure, with computed image backgrounds of different resolution. In doing so, it is possible to detect larger defects, given that each level identifies defects of different sizes. Experimental results presented in this paper are obtained from the industrial automatic quality control system QEyeTunnel employed in the production line at the Mercedes-Benz factory in Vitoria, Spain. A complete analysis of the algorithm performance will be shown here, together with several tests proving the robustness and reliability of our proposal.This work is supported by VALi+d (APOSTD/2016/044) and PROMETEO (PROMETEOII/2014/044) Programs, both from Conselleria d'Educacio, Generalitat Valenciana.Molina, J.; Solanes Galbis, JE.; Arnal-Benedicto, L.; Tornero Montserrat, J. (2017). On the detection of defects on specular car body surfaces. Robotics and Computer-Integrated Manufacturing. 48:263-278. https://doi.org/10.1016/j.rcim.2017.04.009S2632784

Sliding mode control for robust and smooth reference tracking in robot visual servoing

[EN] An approach based on sliding mode is proposed in this work for reference tracking in robot visual servoing. In particular, 2 sliding mode controls are obtained depending on whether joint accelerations or joint jerks are considered as the discontinuous control action. Both sliding mode controls are extensively compared in a 3D-simulated environment with their equivalent well-known continuous controls, which can be found in the literature, to highlight their similarities and differences. The main advantages of the proposed method are smoothness, robustness, and low computational cost. The applicability and robustness of the proposed approach are substantiated by experimental results using a conventional 6R industrial manipulator (KUKA KR 6 R900 sixx [AGILUS]) for positioning and tracking tasks.Spanish Government, Grant/Award Number: BES-2010-038486; Generalitat Valenciana, Grant/Award Number: BEST/2017/029 and APOSTD/2016/044Muñoz-Benavent, P.; Gracia, L.; Solanes, JE.; Esparza, A.; Tornero, J. (2018). Sliding mode control for robust and smooth reference tracking in robot visual servoing. International Journal of Robust and Nonlinear Control. 28(5):1728-1756. https://doi.org/10.1002/rnc.3981S17281756285Hutchinson, S., Hager, G. D., & Corke, P. I. (1996). A tutorial on visual servo control. IEEE Transactions on Robotics and Automation, 12(5), 651-670. doi:10.1109/70.538972Chaumette, F., & Hutchinson, S. (2008). Visual Servoing and Visual Tracking. Springer Handbook of Robotics, 563-583. doi:10.1007/978-3-540-30301-5_25Corke, P. (2011). Robotics, Vision and Control. Springer Tracts in Advanced Robotics. doi:10.1007/978-3-642-20144-8RYAN, E. P., & CORLESS, M. (1984). Ultimate Boundedness and Asymptotic Stability of a Class of Uncertain Dynamical Systems via Continuous and Discontinuous Feedback Control. IMA Journal of Mathematical Control and Information, 1(3), 223-242. doi:10.1093/imamci/1.3.223Chaumette, F., & Hutchinson, S. (2006). Visual servo control. I. Basic approaches. IEEE Robotics & Automation Magazine, 13(4), 82-90. doi:10.1109/mra.2006.250573Chaumette, F., & Hutchinson, S. (2007). Visual servo control. II. Advanced approaches [Tutorial]. IEEE Robotics & Automation Magazine, 14(1), 109-118. doi:10.1109/mra.2007.339609Bonfe M Mainardi E Fantuzzi C Variable structure PID based visual servoing for robotic tracking and manipulation 2002 Lausanne, Switzerland https://doi.org/10.1109/IRDS.2002.1041421Solanes, J. E., Muñoz-Benavent, P., Girbés, V., Armesto, L., & Tornero, J. (2015). On improving robot image-based visual servoing based on dual-rate reference filtering control strategy. Robotica, 34(12), 2842-2859. doi:10.1017/s0263574715000454Elena M Cristiano M Damiano F Bonfe M Variable structure PID controller for cooperative eye-in-hand/eye-to-hand visual servoing 2003 Istanbul, Turkey https://doi.org/10.1109/CCA.2003.1223145Hashimoto, K., Ebine, T., & Kimura, H. (1996). Visual servoing with hand-eye manipulator-optimal control approach. IEEE Transactions on Robotics and Automation, 12(5), 766-774. doi:10.1109/70.538981Chan A Leonard S Croft EA Little JJ Collision-free visual servoing of an eye-in-hand manipulator via constraint-aware planning and control 2011 San Francisco, CA, USA https://doi.org/10.1109/ACC.2011.5991008Allibert, G., Courtial, E., & Chaumette, F. (2010). Visual Servoing via Nonlinear Predictive Control. Lecture Notes in Control and Information Sciences, 375-393. doi:10.1007/978-1-84996-089-2_20Kragic, D., & Christensen, H. I. (2003). Robust Visual Servoing. The International Journal of Robotics Research, 22(10-11), 923-939. doi:10.1177/027836490302210009Mezouar Y Chaumette F Path planning in image space for robust visual servoing 2000 San Francisco, CA, USA https://doi.org/10.1109/ROBOT.2000.846445Morel, G., Zanne, P., & Plestan, F. (2005). Robust visual servoing: bounding the task function tracking errors. IEEE Transactions on Control Systems Technology, 13(6), 998-1009. doi:10.1109/tcst.2005.857409Hammouda, L., Kaaniche, K., Mekki, H., & Chtourou, M. (2015). Robust visual servoing using global features based on random process. International Journal of Computational Vision and Robotics, 5(2), 138. doi:10.1504/ijcvr.2015.068803Yang YX Liu D Liu H Robot-self-learning visual servoing algorithm using neural networks 2002 Beijing, China https://doi.org/10.1109/ICMLC.2002.1174473Sadeghzadeh, M., Calvert, D., & Abdullah, H. A. (2014). Self-Learning Visual Servoing of Robot Manipulator Using Explanation-Based Fuzzy Neural Networks and Q-Learning. Journal of Intelligent & Robotic Systems, 78(1), 83-104. doi:10.1007/s10846-014-0151-5Lee AX Levine S Abbeel P Learning Visual Servoing With Deep Features and Fitted Q-Iteration 2017Fakhry, H. H., & Wilson, W. J. (1996). A modified resolved acceleration controller for position-based visual servoing. Mathematical and Computer Modelling, 24(5-6), 1-9. doi:10.1016/0895-7177(96)00112-4Keshmiri, M., Wen-Fang Xie, & Mohebbi, A. (2014). Augmented Image-Based Visual Servoing of a Manipulator Using Acceleration Command. IEEE Transactions on Industrial Electronics, 61(10), 5444-5452. doi:10.1109/tie.2014.2300048Edwards, C., & Spurgeon, S. (1998). Sliding Mode Control. doi:10.1201/9781498701822Zanne P Morel G Piestan F Robust vision based 3D trajectory tracking using sliding mode control 2000 San Francisco, CA, USAOliveira TR Peixoto AJ Leite AC Hsu L Sliding mode control of uncertain multivariable nonlinear systems applied to uncalibrated robotics visual servoing 2009 St. Louis, MO, USAOliveira, T. R., Leite, A. C., Peixoto, A. J., & Hsu, L. (2014). Overcoming Limitations of Uncalibrated Robotics Visual Servoing by means of Sliding Mode Control and Switching Monitoring Scheme. Asian Journal of Control, 16(3), 752-764. doi:10.1002/asjc.899Li, F., & Xie, H.-L. (2010). Sliding mode variable structure control for visual servoing system. International Journal of Automation and Computing, 7(3), 317-323. doi:10.1007/s11633-010-0509-5Kim J Kim D Choi S Won S Image-based visual servoing using sliding mode control 2006 Busan, South KoreaBurger W Dean-Leon E Cheng G Robust second order sliding mode control for 6D position based visual servoing with a redundant mobile manipulator 2015 Seoul, South KoreaBecerra, H. M., López-Nicolás, G., & Sagüés, C. (2011). A Sliding-Mode-Control Law for Mobile Robots Based on Epipolar Visual Servoing From Three Views. IEEE Transactions on Robotics, 27(1), 175-183. doi:10.1109/tro.2010.2091750Parsapour, M., & Taghirad, H. D. (2015). Kernel-based sliding mode control for visual servoing system. IET Computer Vision, 9(3), 309-320. doi:10.1049/iet-cvi.2013.0310Xin J Ran BJ Ma XM Robot visual sliding mode servoing using SIFT features 2016 Chengdu, ChinaZhao, Y. M., Lin, Y., Xi, F., Guo, S., & Ouyang, P. (2016). Switch-Based Sliding Mode Control for Position-Based Visual Servoing of Robotic Riveting System. Journal of Manufacturing Science and Engineering, 139(4). doi:10.1115/1.4034681Moosavian, S. A. A., & Papadopoulos, E. (2007). Modified transpose Jacobian control of robotic systems. Automatica, 43(7), 1226-1233. doi:10.1016/j.automatica.2006.12.029Sagara, S., & Taira, Y. (2008). Digital control of space robot manipulators with velocity type joint controller using transpose of generalized Jacobian matrix. Artificial Life and Robotics, 13(1), 355-358. doi:10.1007/s10015-008-0584-7Khalaji, A. K., & Moosavian, S. A. A. (2015). Modified transpose Jacobian control of a tractor-trailer wheeled robot. Journal of Mechanical Science and Technology, 29(9), 3961-3969. doi:10.1007/s12206-015-0841-3Utkin, V., Guldner, J., & Shi, J. (2017). Sliding Mode Control in Electro-Mechanical Systems. doi:10.1201/9781420065619Utkin, V. (2016). Discussion Aspects of High-Order Sliding Mode Control. IEEE Transactions on Automatic Control, 61(3), 829-833. doi:10.1109/tac.2015.2450571Romdhane, H., Dehri, K., & Nouri, A. S. (2016). Discrete second-order sliding mode control based on optimal sliding function vector for multivariable systems with input-output representation. International Journal of Robust and Nonlinear Control, 26(17), 3806-3830. doi:10.1002/rnc.3536Sharma, N. K., & Janardhanan, S. (2017). Optimal discrete higher-order sliding mode control of uncertain LTI systems with partial state information. International Journal of Robust and Nonlinear Control. doi:10.1002/rnc.3785LEVANT, A. (1993). Sliding order and sliding accuracy in sliding mode control. International Journal of Control, 58(6), 1247-1263. doi:10.1080/00207179308923053Levant, A. (2003). Higher-order sliding modes, differentiation and output-feedback control. International Journal of Control, 76(9-10), 924-941. doi:10.1080/0020717031000099029Bartolini, G., Ferrara, A., & Usai, E. (1998). Chattering avoidance by second-order sliding mode control. IEEE Transactions on Automatic Control, 43(2), 241-246. doi:10.1109/9.661074Siciliano, B., Sciavicco, L., Villani, L., & Oriolo, G. (2009). Robotics. Advanced Textbooks in Control and Signal Processing. doi:10.1007/978-1-84628-642-1Deo, A. S., & Walker, I. D. (1995). Overview of damped least-squares methods for inverse kinematics of robot manipulators. Journal of Intelligent & Robotic Systems, 14(1), 43-68. doi:10.1007/bf01254007WHEELER, G., SU, C.-Y., & STEPANENKO, Y. (1998). A Sliding Mode Controller with Improved Adaptation Laws for the Upper Bounds on the Norm of Uncertainties. Automatica, 34(12), 1657-1661. doi:10.1016/s0005-1098(98)80024-1Yu-Sheng Lu. (2009). Sliding-Mode Disturbance Observer With Switching-Gain Adaptation and Its Application to Optical Disk Drives. IEEE Transactions on Industrial Electronics, 56(9), 3743-3750. doi:10.1109/tie.2009.2025719Chen, X., Shen, W., Cao, Z., & Kapoor, A. (2014). A novel approach for state of charge estimation based on adaptive switching gain sliding mode observer in electric vehicles. Journal of Power Sources, 246, 667-678. doi:10.1016/j.jpowsour.2013.08.039Cong, B. L., Chen, Z., & Liu, X. D. (2012). On adaptive sliding mode control without switching gain overestimation. International Journal of Robust and Nonlinear Control, 24(3), 515-531. doi:10.1002/rnc.2902Taleb, M., Plestan, F., & Bououlid, B. (2014). An adaptive solution for robust control based on integral high-order sliding mode concept. International Journal of Robust and Nonlinear Control, 25(8), 1201-1213. doi:10.1002/rnc.3135Zhu, J., & Khayati, K. (2016). On a new adaptive sliding mode control for MIMO nonlinear systems with uncertainties of unknown bounds. International Journal of Robust and Nonlinear Control, 27(6), 942-962. doi:10.1002/rnc.3608Hafez AHA Cervera E Jawahar CV Hybrid visual servoing by boosting IBVS and PBVS 2008 Damascus, SyriaKermorgant O Chaumette F Combining IBVS and PBVS to ensure the visibility constraint 2011 San Francisco, CA, USACorke, P. I., & Hutchinson, S. A. (2001). A new partitioned approach to image-based visual servo control. IEEE Transactions on Robotics and Automation, 17(4), 507-515. doi:10.1109/70.954764Yang, Z., & Shen, S. (2017). Monocular Visual–Inertial State Estimation With Online Initialization and Camera–IMU Extrinsic Calibration. IEEE Transactions on Automation Science and Engineering, 14(1), 39-51. doi:10.1109/tase.2016.2550621Chesi G Hashimoto K Static-eye against hand-eye visual servoing 2002 Las Vegas, NV, USABourdis N Marraud D Sahbi H Camera pose estimation using visual servoing for aerial video change detection 2012 Munich, GermanyShademan A Janabi-Sharifi F Sensitivity analysis of EKF and iterated EKF pose estimation for position-based visual servoing 2005 USAMalis, E., Mezouar, Y., & Rives, P. (2010). Robustness of Image-Based Visual Servoing With a Calibrated Camera in the Presence of Uncertainties in the Three-Dimensional Structure. IEEE Transactions on Robotics, 26(1), 112-120. doi:10.1109/tro.2009.2033332Chen J Behal A Dawson D Dixon W Adaptive visual servoing in the presence of intrinsic calibration uncertainty 2003 USAMezouar Y Malis E Robustness of central catadioptric image-based visual servoing to uncertainties on 3D parameters 2004 Sendai, JapanMarchand, E., Spindler, F., & Chaumette, F. (2005). ViSP for visual servoing: a generic software platform with a wide class of robot control skills. IEEE Robotics & Automation Magazine, 12(4), 40-52. doi:10.1109/mra.2005.157702

Detecting dings and dents on specular car body surfaces based on optical flow

[EN] This paper introduces a new approach to detect defects cataloged as dings and dents on car body surfaces, which is currently one of the most important issues facing quality control in the automotive industry. Using well-known optical flow algorithms and the deflectometry principle, the method proposed in this work is able to detect all kind of anomalies on specular surfaces. Hence, our method consists of two main steps: first, in the pre-processing step, light patterns projected on the body surface sweep uniformly the area of inspection, whilst a new image fusion law, based on optical flow, is used to obtain a resulting fused image holding the information of all variations suffered by the projected patterns during the sweeping process, indicating the presence of anomalies; second, a new post-processing step is proposed that avoids the need of using pre-computed reference backgrounds in order to differentiate defects from other body features such as style-lines. To that end, the image background of the resulting fused image is estimated in the first place through a method based on blurring the image according to the direction of each pixel. Afterwards, the estimated image background is used in a new subtraction law through which defects are well differentiated from other surface deformations, allowing the detection of defects in the entire illuminated area. In addition, since our approach, together with the system used, computes defects in less than 15 s, it satisfies the assembly plants time requirements. Experimental results presented in this paper are obtained from the industrial automatic quality control system QEyeTunnel employed in the production line at the Mercedes-Benz factory in Vitoria, Spain. A complete analysis of the algorithm performance will be shown here, together with several tests proving the robustness and reliability of our proposal.This work is supported by VALi+d (APOSTD/2016/044) and PROMETEO (PROMETEOII/2014/044) Programs, both from Conselleria d'Educacio, Generalitat Valenciana.Arnal-Benedicto, L.; Solanes Galbis, JE.; Molina, J.; Tornero Montserrat, J. (2017). Detecting dings and dents on specular car body surfaces based on optical flow. Journal of Manufacturing Systems. 45:306-321. https://doi.org/10.1016/j.jmsy.2017.07.006S3063214

An Active Safety System for Low-Speed Bus Braking Assistance

Accidents in which buses or coaches are involved cause thousands of injuries and fatalities every year. To reduce their number and severity, the paper describes an Advanced Driver Assistance Systems (ADAS) based on a haptic throttle pedal and emergency braking. It also proposes a computationally efficient algorithm with a methodology based on three main concepts: a simplified but accurate vehicle model; an efficient collision detection system considering driver's intention and pedestrians wandering around the vehicle; and a risk evaluation system to generate warnings and emergency braking signals. Finally, the performance of the proposed ADAS is validated using a driving simulation cabin with a very realistic urban scenario and original elements from real buses. The results show a statistically significant improvement in safety, as the number of collisions and high risk situations are clearly minimized, reaction time to press the brake pedal is improved and time to collision increased in emergency situations. Implementation of the proposed ADAS into city buses would potentially improve safety, reducing the frequency and severity of accidents with pedestrians.This work was supported in part by Ministry of Science and Innovation of Spain through the SAFEBUS Project "Sistemas Avanzados de Seguridad Integral en Autobuses" under Grant IPT-2011-1165-370000 and the SAFETRANS Project "Sistemas de Conduccion Segura de Vehiculos de Transporte de Pasajeros y Materiales con Asistencia Haptica/Audiovisual e Interfaces Biomedicas" under Grant DPI2013-42302-R and in part by the Generalitat Valenciana, Programa VALi+d (ACIF/2010/206). The Associate Editor for this paper was E. Kosmatopoulos.Girbés, V.; Armesto Ángel, L.; Dols Ruiz, JF.; Tornero Montserrat, J. (2017). An Active Safety System for Low-Speed Bus Braking Assistance. IEEE Transactions on Intelligent Transportation Systems. 18(2):377-387. https://doi.org/10.1109/TITS.2016.2573921S37738718

Combining haptics and inertial motion capture to enhance remote control of a dual-arm robot

High dexterity is required in tasks in which there is contact between objects, such as surface conditioning (wiping, polishing, scuffing, sanding, etc.), specially when the location of the objects involved is unknown or highly inaccurate because they are moving, like a car body in automotive industry lines. These applications require the human adaptability and the robot accuracy. However, sharing the same workspace is not possible in most cases due to safety issues. Hence, a multi-modal teleoperation system combining haptics and an inertial motion capture system is introduced in this work. The human operator gets the sense of touch thanks to haptic feedback, whereas using the motion capture device allows more naturalistic movements. Visual feedback assistance is also introduced to enhance immersion. A Baxter dual-arm robot is used to offer more flexibility and manoeuvrability, allowing to perform two independent operations simultaneously. Several tests have been carried out to assess the proposed system. As it is shown by the experimental results, the task duration is reduced and the overall performance improves thanks to the proposed teleoperation method